Conventionally, there has been a technique for generating a multiple-viewpoint image formed by imaging parts of light that have passed through different regions of the lens by dividing lenses or apertures of an imaging optical system or pixels in an image sensor. This technique is configured to calculate the phase difference between multiple-viewpoint images by searching the multiple-viewpoint images for pixels in which the same object is captured, so that the distance to the object captured in the pixels is found from the phase difference. Different multiple-viewpoint images are different in blur shape (point spread function). For this reason, in the technique above, a blur in an image is corrected into a natural blur having a circular or like shape, based on the calculated phase difference.
However, the above-mentioned technique is inconvenient in that highly accurate distance measurement is difficult. Specifically, in the conventional technique, highly accurate distance measurement is difficult because different point spread functions are used for the respective multiple-viewpoint images in calculating degrees of coincidence for calculating the phase difference.